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Abstract 

We provide a full characterization of the oblique projector f/(yf/) ' V in the general case where 
the range of U and the null space of V are not complementary subspaces. We discuss the new 
result in the context of constrained least squares minimization which finds many applications in 
engineering and statistics. 
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1. Introduction 

Let E e C'"^™ be idempotent, - E. The null space and range of any idempotent matrix 
are complementary, cf. [1, Theorem 2.8], 

R{E) + N(E) = C"\R(E) n NiE) = {0}, 

and we say that E is an oblique projector onto R{E) along N{E). For any two complementary 
subspaces of C" we denote the oblique projector onto L along M by Pl,m- The orthogonal pro- 
jector onto L is denoted by f ^ :- Pl,l^, where is the orthogonal complement of L. Oblique 
projectors arise in numerous engineering and statistical applications, see |1, Chapter 8], |2] and 
references therein. Many of their properties follow from the general solution to the matrix equa- 
tion XAX - X studied in 1960-ies in the context of the various pseudoinverses, cf. [3]. This 
literature is mature, with excellent monographs such as [1]. In particular it is very well under- 
stood how to construct an oblique projector with a prescribed range and null space. 

Proposition 1.1. Let L, M be complementary subspaces ofC". For any two matrices U, V with 
R{U) = L and NiV) = M one has 

where the superscript " t " denotes the Moore-Penrose inverse. If U and V are in addition or- 
thogonal projectors (i.e. they are Hermitian and idempotent) one obtains an even simpler form 
due to Greville (3.1) and Theorem 2], 

Pl,M = PLiPM-PlYPM- = (Pm-Pl)^'- (1) 



I would like to thank Adi Ben-Israel, Laura Rebollo-Neira, Gotz Trenkler and two anonymous referees for helpful 
comments and pointers to references. Any eiTors in the manuscript are my responsibility. 

Preprint submitted to Linear Algebra and Its Applications May 26, 2009 



The converse problem of characterizing the range and null space of a given idempotent matrix 
has not received the same amount of attention. The motivation for studying idempotents of 
the form U{VU)'^V in the general case where R{U) + N{V) C C" and/or R{U) n N(V) + {0} 
comes, among others, from constrained least squares optimization with a range of applications 
mentioned above. Briefly, the problem 

min ||Aix - ^ill^ , subject to A2X = ^72, 

.reC" 

gives rise to the projector D2{A\D2)^ A\ where D2 is an arbitrary but fixed matrix with the 
property R{D2) - N(A2). In this situation we typically have neither R(D2) + N(Ai) - C"' nor 
R(D2) n N(Ai) = {0). Oblique projectors of the form U(VUyV with R(U) + NiV) = C" and 
R(U) n N(V) + (0) feature also in signal reconstruction, cf. 

Given that U(VU) ' V has a wide range of applications it is desirable to understand its geo- 
metric nature. One might conjecture that in general 

UiVUy^V = Pl,m, where (2) 
L = PumNivy = RiU) n (R{U) n N(V)y, (3) 
M = NiV) + iN(V) + R(U))-^, (4) 

but the behaviour of the projector is somewhat more intricate and cannot be described based on 
the knowledge of RiU) and N{V) alone. The conjecture ©-(Hli turns out to be true only when 
both U and V are orthogonal projectors. Surprisingly, the main tool in proving the general result 
is the Zlobec formula [6J in conjunction with Proposition ll.il 

The result presented here is different from the problem discussed by Rao and Yanai [7] in 
which projectors onto and along two given subspaces are considered under the assumption that 
the subspaces are not necessarily spanning the whole space. In such a situation, the projector no 
longer needs to be idempotent. 

The paper is organized as follows. In section 2 we introduce required terminology and nota- 
tion, we establish the main tools and prove Proposition ll.il In section 3 we state and prove the 
main result. In section 4 we discuss application of the main result to constrained least squares 
minimization and the link to the minimal norm solution of Elden jQ]- 

2. Preliminaries 

We use notation of fl*]. A* denotes the conjugate transpose of matrix A. We write r(A), R(A), 
N(A) for the rank, range and null space of A, respectively. Consider the following relations 



AXA=A, (I.l) 

XAX = X, (1.2) 

AX = (AXy, (1.3) 

XA = (XAy. (1.4) 



We write X E A{i, j, . . . , k], if X satisfies conditions (I./), (l.j), . . . , (l-k). A ' denotes the Moore- 
Penrose inverse which is the unique element of A{1, 2, 3,4). The following theorem is our main 
tool. 
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Theorem 2.1 (id Theorem 2.13]). Let A e C"'^", U e C"^ ', V e C'^™ and 

Z = UiVAUf>V, 

where (VAf/)^'' is a fixed but arbitrary element of(VAU){l]. Then 
a}Ze A|l) if and only ifr(VAU) = r(A); 
b)Ze A{2) andR(Z) = if and only ifr(VAU) = r(f7); 
cj Z € A{2) andN(Z) = A^(y) i/ W onfy ifr(VAU) = r(y); 

(12) ~ ~ ~ ~ ('12') 

d) Z - A'^^' if and only ifr(U) - r(V) - r(VAU) - r(A), where A 'J is the unique 

R(U),N(V) ■' J J ^ ' I V R(U),N(V) 

element of A[\, 2} with range R(U) and null space NiV), also known as the oblique pseudoinverse 
(cf [9]). 

Corollary 2.2. The Zlobec formula l^, 

A^ = A*(A*AA*y'^^A*, (5) 

is now obtained by setting f7 = V = A* in part d) and arguing A^^^^l^ ^^^.^ = A ' . 

The following is a pre-cursor to the main result in this note. The "if part appears, for example, 
in 10(3.51)]. 

Corollary 2.3. U(VU)'-^^V = Pr0in(V) if and only ifr{W) = r{V) = r{U). 

Next we show that the form U{VUyV covers all idempotent matrices. 

Lemma 2.4. Let U e C'"'^'', V e C""". R{U) andN(V) are complementary subspaces ofC"' if 
and only ifr(U) = r(V) = r(VU). 

Proof. If: By Corollary O ^(Vt/)^^ = Pr(U),n(V) which impHes that R(U),N(V) are comple- 
mentary. 

Only if: i) complementarity implies dim(R(U)) + dim(A^(y)) - m. On rearranging we obtain 
r{U) - m - dim(A^(y)) and by the rank-nullity theorem r{U) - r(V). 

ii) Complementarity also impHes R(U) n N(V) - {0} which yields N(VU) - N(U). By 
rank-nullity theorem we obtain r(VU) = r{U) . □ 

Proposition 2.5. Matrix E e C™^'" is idempotent if and only if there are matrices U e C'"^'', V e 
C^'" such that 

E := U(VUyV. (6) 
Proof. The 'if statement follows easily from Q and (1.2), 

£2 ^ u(vuyvu(vu)W = E. 

The 'only if part: construct U so that its columns form a basis of R(E) and construct V* so 
that its columns form the basis of A^(£')^. This impHes R{U) = R{E),N(V) = N(E). Since E 
is idempotent R(U),N(V) are by construction complementary and from Lemma 1241 we obtain 
r(U) = r(V) = r(VU). By Corollary|23]f/(V;7)' V = Pr(E),n(E) ^ E. □ 

Remark 2.6. A comprehensive characterization of projectors appears in Proposition 12.51 
resembles a result ofMitra hlR Theorem 3a] who shows that all idempotent matrices are of the 
form U{VUf-^>V where (yf/)*''^' is an arbitrary element ofVU{l,2}. This result is generalized 
further in Theorem 2.13] to the form U(VUY^^V, see Corollarv \2.3\ Proposition \2.5\ soes in 
the opposite direction in order to avoid the ambiguity associated with {\,2}-inverses. 
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To conclude we provide a proof of Proposition 11.11 

Proof (Proposition [TTT]i. The first statement follows from the 'only if part in the proof of Propo- 
sition J2.5l The second part follows from identities (Pm^Pl)^ - Pl{Pm^Pl)^ = {Pm^Pl)^ Pm^, 
see 111 Exercise 2.57]. □ 



3. Result 

Theorem 3.1. Given two arbitrary matrices U e C"""'', V e C""" the matrix E = U{VU)^V is 
idempotent with range and null space given by 

R(E) = R(UU*V*) = RiUWVV) = R(U) n ((UU*y'(R(U) n N(V)))^, (7) 

N{E) = mWVV) = NiUWVV) = N(V) e (VV)"'" + N(V)y. (8) 

Proof. By Zlobec's formula (|5]l with A - VU we obtain 

E = uu*v*{u*vvuu*vy'^^u''Vv. 

Setting U = UU*V*, V = U*V*V we claim r{U) = r(y) = r{VU) = r(yf/). Indeed, 

r(vu) = r(yf/f/*y*) = r(yf/f/*y*y;7f/*y*) < riu'WuVV) = r(yf7), (9) 

riVU) < r(U) = r(f/f/*y*) < r(U*V*) = r(y;7), (10) 
riVU) < r(V) = r(f/*y*y) < r(U*V*) = r(yt/). (11) 

CorollaryOyields R{E) = RiU),N{E) = A?(y). From 

r(yf/) = r(yf/f/*y*) = r(yf/;7*y*yf/;7*y*) < r(f/;7*y*y) < r(u*v*) = r(yf/), 

and from we obtain r(yf/) = r{UU*V*) = r(;7f/*y*y) which impHes R{UU*V*) = 

7;([/[/*yy). The proof of A^(f/*y*y) = A^( f/ ?/* y*y) proceeds similarly by showing r(f/*y*y) = 
r(UU*V*V). 

To show the last equaUty in (O we observe C" = A^(y) e R(V*). Since A^(y) c N(U*VV) 
we have 

A^(f/*yy) = my) e (/?(y) n A^(i/*y*y)). (12) 

Continuing with the second term on the right hand side we obtain 

yeR(V*)nN(U*VV) <^ (V*Vy e N(U*) n RiV*)) A (y e RiV)) 
^ ye (V*V)HN(U*) n R(V*)), 

which yields 

R(V') n N{U*v*v) = (v*v)HR(U)^ n n(V)^) = (V'vy(R(U) + mv))-^. (13) 

On substituting ( fT3] l into (fT2l) we obtain the desired result. 

The last equality in (|7]) is obtained by writing R{UU*V*) - N{VUU*)^ and then evaluating 
N{VUU*) by exchanging the role of U and V* in ([T2li and O. □ 



Remark 3.2. Special cases of Theorem \3.1\ include situations covered by Corollarv \2.3\ in which 
r(U) = r(V) = r(VU) and we have R(E) = R(U),N(E) = N(V); the Langenhop form Ul 
Lemma 2.2] with VU — I is a case in point. The Grevi lie formula (O also falls into this category. 
Hirabayashi and Unser [5. Lemma 3] encounter the case R(U) + N(V) — C" and R{U)r\N(V) + 
{0}, yielding R(E) = R(UU*V*),N(E) = N(V). 
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4. Application 

Proposition 4.1. Let Ai e C"""\bi e C"\A2 e C*^",r(A2) ^ k > l,b2 e C*. Solutions of the 
problem 

min||Aix - /jill^ , subject to A2X - b2, (14) 

.veC" 

lie in the set 

E = {Z)2(AiD2)"'"AiA>i + (/ - D2(A,D2)Ui)(A^/72 + z) : z e MA2)}, (15) 

where D2 is an arbitrary but fixed matrix with the property R(D2) — N(A2). 

Proof. See | 1, Exercise 3.10]. □ 

In general, the projector D2(AiD2)' Ai will depend on how D2 is chosen. However, Theorem 
l3.1l shows that there is a special case when D2(AiD2) ' Ai is actually invariant to the choice of D2. 



Corollary 4.2. Using the notation of Proposition \4~l\ assume further r{A\) — n. Then 

Z)2(AiD2)'Ai - PN{A2)XA\Ai)-^R(Al), 

and H is a singleton, 

S = {A"[fo, +(AJAi)-'A*(A2(A;Ai)-'A*)-'(/72 -A2A"j;^i)). 

Proof. We have N{Ai) - and by Theorem l3.ll 

i?(D2(AiD2)'"Ai) = R{D2) n ({0})^ = A^(A2), 
A^(D2(AiD2)"'"Ai) = {A\A,r'R{D2t = (A;Ai)-';;(A;). 

This implies (/ - D2(AiD2)' Ai)z = for all z e A^(A2) and by Proposition O 

(/-D2(AiD2)Ai) = (A;Ai)-'a;(A2(AJAi)"'A-)-1A2. 

The rest follows from Proposition l4.1l □ 



Note that Corollarv 14.21 is not covered by Corollarv 12.31 since n - k — r{D2) — r(A 1D2) < 
r(Ai) - n. In situations where the choice of D2 impacts on the projector D2(AiD2)^Ai Theorem 
13. II guides us to the convenient choice of D2 which simplifies the geometry of the result and also 
helps to identify the element of H with minimal distance from a given reference point. 



Corollary 4.3. Using the notation of Proposition \4J\ the following statements hold: 
1 . The constrained least squares minimizer in (\14\l lies in the set 



with 



E = {Albi + Pii,x(Alb2 - A\bi) +z:ze N(Ai) n N(A2)}, (16) 



Px,y = / - Py,x = (Aid - A;A2))"'"Ai, (17) 

X = PN(A,)RiAl) = A^(A2) n iN{A2) n NiAi))^, (18) 

J/ = N(Ai) e (AlAiYiMAi) + N(A2)y. (19) 
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2. The element ofB. with the smallest Euclidean norm is given by 

^■.^A\bi+Py,x{Alb2-A\bi). 

3. For any y € C" the solution q/mirivEH 11-"^ — y II is given by 

ip{y) := ^ + PN(Aor,N(A2)y- (20) 

Proof. 1. On setting D2 - I - AjAa = Pniaj) Proposition ^. 1 l and Theorem l3. ll vield 

H = A\bi + P^,x(.A[b2 - A>i + N{A2)), (21) 

with Px,}i, X and J/ given in O-CSll. From ^ we obtain N(A2) = ,Ye(A^(Ai) n A^(A2)) which 
impHes 

P}l,xN{A2) = P^,,xiN{A,) n N{A2)) = A?(Ai) n A?(A2), (22) 
the last equality following from N{A\) n A^(A2) c J/. Substitution of (|22]l into (|2T]> yields ( fT6b . 

2. By ([T8]l we have c (A^(A2) n N{Ai)Y = «(A*) + R{A*^. Consequently 

Pi,ARiA\) + R{Al)) = (/ - Px,y){R{A\) + R{Al)) c R{A\) + R(A;). (23) 

This implies 

^ 6 /?(Ap + R(Al) = (A^(A2) n N(Ai))\ (24) 
By ( fT6] l X - f € N(Ai) n A^(A2) for any x e H which together with (l24l l yields 

||x f = ||x - ^ + ^ ||2 = ||x -^f + 11^ ||2 for all x 6 H. 

3. By (dill, (|20li and (|24| we obtain x - t//iy) e A^(A2) n N(Ai) and (/^(y) -ye (A^(A2) n A^(Ai))-^ 
which implies ||x -y\\^ - ||x - 1/^(3;) + ij/{y) - y\\^ - ||x - ^ |p + ||^ - y |p, for all x e H. □ 

Remark 4.4. It is well known that vector Aj^i has the smallest Euclidean norm among all 
solutions of the unconstrained least squares problem minj^gc" ||Aix - fei||. We have shown in part 
2. of Corollar\ \4.3\ that £ — A^^b\ + Py^x{A2b2 — A^^b\) is the shortest solution of the constrained 
least squares problem Iil4\l . 

Elden ^3 Theorem 2.1] studied minimal norm solutions of constrained least squares. On 
setting 

h = b2-A2A\bu f = x-A[bu K = A^ L ^ A2, M = I, 
Elden 's solution yields that 

( := A\bi + (/ - PN(AAAiPmM))^Ai)Al{b2-A2A\bi) 

minimizes the Euclidean distance ||x - A[/7i|| among all constrained minimizers x e H. 

With a little bit of work one finds — ^ — P\)xPN{A2)A\b\ — ^, since f a'(A2)A| & X by virtue 
ofilSll. Thus part 3. of Corollar\ \4.3\ simplifies and extends Elden's result. 
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